UNIQUENESS OF FOURIER-JACOBI MODELS: THE 
ARCHIMEDEAN CASE 



YIFENG LIU AND BINYONG SUN 

Abstract. We prove uniqueness of Fourier- Jacobi models for general linear groups, 
unitary groups, symplectic groups and metaplectic groups, over an archimedean local 
field. 



1. Introduction and the main result 

Uniqueness of Bessel models and Fourier- Jacobi models is the basic starting point 
to study L- functions for classical groups by Rankin-Selberg method ([7,8]). Break- 
throughs have been made towards the proof of the uniqueness in the recent years. 
Over a non-archimedean local field of characteristic zero, uniqueness of Bessel mod- 
els and Fourier-Jacobi models is now completely proved, by the works of Aizenbud- 
Gourevitch-Rallis-Schiffmann [1], Sun [19], Waldspurger [23] and Gan-Gross-Prasad 
[6] . Over an archimedean local field, uniqueness of Bessel models is proved by Jiang- 
Sun-Zhu [11]. Only uniqueness of Fourier-Jacobi models in the archimedean case 
remains open. This article is aimed to prove this remaining case. 

Theorem A. Let G be a classical group GL n (R), GL n (C), U(p, q), Sp 2m (C) ; or a 
metaplectic group Sp 2m (R), with an r-th Fourier-Jacobi subgroup S r of it (r > 1, n > 
2r, p,q > r, m > r). Denote by J r and N r the Jacobi quotient and the Whittaker quo- 
tient of S r , respectively. Then for every irreducible Casselman-Wallach representation 
n of G, every nondegenerate irreducible Casselman-Wallach representation a of J r , 
and every nondegenerate unitary character ip of N r , one has that 

dim Horn 5 r (7r£g><7, ?/>) < 1. 

Here a and ip are viewed as representations of S r via inflations, and "(g)" stands for 
the completed projective tensor product. By abuse of notation, we do not distinguish 
representations with their underlying vector spaces. Fourier-Jacobi subgroups as well 
as their Jacobi quotients and Whittaker quotients are defined in Section 2. The notion 
of "nondegenerate unitary character on iV r " is also explained in Section 2. The notions 
concerning Casselman-Wallach representations are explained in Section 3.3. Note that 
Theorem A for Sp 2m (M) implies the analogous result for the symplectic group Sp 2m (IR). 

When n = 2r, or p = q = r, or m = r, Theorem A asserts uniqueness of Whittaker 
models for G. See [15], [4] for uniqueness of Whittaker models for quasi-split linear 
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groups over R (or [11] for a quick proof). When G = U(n, 1) and 7r is unitary, Theorem 
A is proved in [2]. 

As in the proof of uniqueness of Bessel models, our idea is to reduce Theorem A to 
the following basic case, which is called the multiplicity one theorem for Fourier-Jacobi 
models. 

Theorem B. Let J be one of the following Jacobi groups 

(1) H 2n+1 (M) x GL n (R), H 2n+1 (C) x GL„(C), H 2pW1 (M) x XJ(p,q), 

H 2n+1 (C) x Sp 2n (C), H 2n+1 (R) x §5 2n (R), p,q,n > 0, 

where "R^k+i" indicates the appropriate Heisenberg group of dimension 2k + 1. Denote 
by G its respective subgroup 

GL n (R), GL n (C), XJ(p,q), Sp 2n (C), §5 2n (R). 

Then for every nondegenerate irreducible Casselman-Wallach representation p of J, 
and every irreducible Casselman-Wallach representation of tt of G, one has that 

dimHom G (p®7r, C) < I. 

In the above inequality, C stands for the trivial representation of G. Theorem B is 
proved in [21], except for the case of G = Sp 2n (R). But in this case, by the classification 
of nondegenerate irreducible Casselman-Wallach representations of Jacobi groups (see 
Section 3.3), Theorem B is obviously equivalent to the analogous result for Sp 2n (R), 
which is also proved in [21]. 

2. Fourier-Jacobi subgroups 

In order to prove Theorem A uniformly, we introduce the following notation. Let 
(IK, l) be one of the followings five IR-algebras with involutions: 

(2) (R x R, t R ) , (CxCVc), (C,-), (R,1 R ), (C, l c ), 

where tR and tc are the maps interchanging the coordinates, 1r and lc are the identity 
maps, and " — " is the complex conjugation. Let £ be a skew-Hermitian K-module; 
namely, it is a free K-module of finite rank, equipped with a nondegenerate IR-bilinear 
map 

(, ) E - E x E ->■ K 

satisfying 

(u, v )e = — (v , u) L E , (au,v) e = a{u,v) e, aeK,u,veE. 

Denote by U(£J) the group of all K-module automorphisms of E which preserve the 
form ( , )e- According to the five cases of (K, t) in (2), it is respectively a real general 
linear group, a complex general linear group, a real unitary group, a real symplectic 
group, or a complex symplectic group. Put 

n , (E) . = fSp» ifK = R; 

I U(E) otherwise, 
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where Sp(E) denotes the metaplectic double cover of the symplectic group Sp(E). 
Then we have a short exact sequence 



(3) 

where 



1 /i K -> U'(E) -> U(£) ->• 1, 



{±1} ifK = R; 
{1} otherwise. 

Let r > 1 and assume that there is a sequence 



5": = I Cl 1 C-Cl r 



i, i 



0,1 r. Put 



of totally isotropic free K-submodules of E so that ranks (Xi 

h(E) := {g G XJ(E) \ (g - l)X t C X^, i = 1, 2 . . . , r}. 

Denote by Jj{E) the inverse image of J?(E) under the covering map \J'(E) — > U{E). 
It is called an r-th Fourier- Jacobi subgroup of XJ'(E). 
When r = 1, we put 

J x (E):=Jj(E) = {geXJ(E)\(g-l)XcX} and J' X (E) := j'^E). 

Then J' X (E) is isomorphic to a Jacobi group of Theorem B, and conversely, all Jacobi 
groups of Theorem B are isomorphic to some Z X (E) (cf. [19, Section 1]). 
For every subset S of E, set 

S ± :={veE \ (v,u) E = for all u e S}. 
Then E' := X^_ 1 / X r -i is obviously a skew-Hermitian K-module. Put 

X \= X r /X r _i c E , 

which is a totally isotropic free K-submodule of E 1 of rank 1. Restrictions yield a 
surjective homomorphism 

There is a unique surjective homomorphism 

£: UE)^J' X >{E') 

so that the squares in 

1 ^ K J'^E) h(E) 1 



1 



J'J 



J' X ,(E>) 



1 



are commutative. In view of the homomorphism jj, we call J' X ,(E') the Jacobi quotient 
ofJ'^E). 
Put 

N?(X) := {g e GL(X) | {g - l)X t C I w , i = 1, 2, . . . , r}. 
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It is a maximal unipotent subgroup of the group GL(X) of K-linear automorphisms of 
X. Restrictions yield a surjective homomorphism %: Jj(-E') — > Ng-(X). Composing 
it with the covering map Jy(-E') —> J?(E), we get a homomorphism 

w' s : 4(£)^N r (X). 

In view of this homomorphism, we call Nj(X) the Whittaker quotient of the Fourier- 
Jacobi subgroup J^-E). 

We review the notion of nondegenerate characters on Ny(Jf). Define a surjective 
homomorphism 

(4) Nj{X) -> A 9 (X) := YlUom^Xi+jX^Xi/X^), g ^ (a 1} a 2 , . . . , a r _ x ), 

where dj is the map v + \-> (g — l)v + Xi-\. Then every unitary character ipsss^x) 
on N^X) descends to a character ipA,j(x) on A$(X) though (4). Note that Agr(X) is a 
free K r_1 -module of rank 1. We say that ^jwx) is nondegenerate if the restriction of 
ipAj-(x) to every nonzero !K r_1 -submodule of Ay(X) is nontrivial. 

3. Preliminaries 

3.1. Almost linear Nash groups. We work in the setting of Nash groups. The 
reader is referred to [16, 17] for details. By a Nash group, we mean a group which 
is simultaneously a Nash manifold so that all group operations (the multiplication 
and the inversion) are Nash maps. Every semialgebraic subgroup of a Nash group is 
automatically closed and is called a Nash subgroup. It is canonically a Nash group. 

A finite dimensional real representation of a Nash group G is said to be a Nash 
representation if the action map G x Vr — > is Nash. A Nash group is said to be 
almost linear if it admits a Nash representation with finite kernel. For every linear 
algebraic group G defined over K, every finite fold topological group cover of an open 
subgroup of G(R) is naturally an almost linear Nash group. On the other hand, every 
almost linear Nash group is of this form. In particular, all groups which occur in last 
section are almost linear Nash groups. 

A Nash group is said to be unipotent if it admits a faithful Nash representation so that 
all group elements act as unipotent linear operators. It follows from the corresponding 
result for linear algebraic groups that every almost linear Nash group has a unipotent 
radical, namely, the largest unipotent normal Nash subgroup. A reductive Nash group 
is defined to be an almost linear Nash group with trivial unipotent radical. 

Recall that a Nash manifold is said to be affine if it is Nash diffeomorphic to a closed 
Nash submanifold of some lR n . Since every finite fold topological cover of an affine Nash 
manifold is an affine Nash manifold, all almost linear Nash groups are affine as Nash 
manifolds. 

3.2. Schwartz inductions. If M is an affine Nash manifold and Vq is a (complex) 
Frechet space, then a V - valued smooth function / G C°°(M; Vo) is said to be Schwartz 
if 

|/| D) „ := sup |(D/)(ac)|„ < oo 

x€M 
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for all Nash differential operator D on M, and all continuous seminorm | • \ u on Vo- 
Recall that a differential operator D on M is said to be Nash if Dip is a Nash function 
whenever ip is a (complex valued) Nash function on M. Denote by C f (M; Vo) C 
C°°(M; V ) the space of Schwartz functions. Then both C'(M; V ) and C°°(M; V ) are 
naturally Frechet spaces, and the inclusion map C ? (M; Vo) c ->- C°°(M; Vo) is continuous. 
Furthermore, we have that (cf. [22, page 533]) 

C ? (M; Vo) = C"(M)®V and C°°(M; Vo) = C°°(M)®Vo, 

where C ? (M) := C"(M;C) and C°°(M) := C°°(M;C). 

Now we recall Schwartz inductions from [5, Section 2]. Let G be an almost linear 
Nash group. Then it is affine as a Nash manifold. Let S be a Nash subgroup of G, and 
let Vo be a smooth Frechet representation of S of moderate growth (cf. [5, Definition 
1.4.1] or [20, Section 2]). Define a continuous linear map 

^ Is,v : C<(G;V ) C°°(G;V ), 

1 J / H- / fl a./(a-^)d»), 

where ds is a left invariant Haar measure on S. We define the unnormalized Schwartz 
induction IndfVo to be the image of the map (5). Under the quotient topology of 
C s (Cr; Vo) and under right translations, it is a smooth Frechet representation of G of 
moderate growth. 

A partition of unity argument shows the following 

Lemma 3.1. Let f G C°°(G; V ). // 

f(sg) = s.f(g), seS, g E G, 

and f is compactly supported modulo S (that is, the support of f has compact image 
under the map G -> S\G), then f e IndfKi- 

If S' is a Nash subgroup of G containing S, then we have a canonical isomorphism 
of representations of G ([5, Lemma 2.1.6]): 

(6) Indf,(Ind|V ) = IndfVo- 
We will use the following result. 

Lemma 3.2. Let Vq and V be smooth moderate growth Frechet representations of S 
and G, respectively. If V is nuclear, then there is an isomorphism of representations 
ofG: 

(7) Indf (y ®(y\ s )) = (IndgV )®V. 
Proof. Note that the diagram 

(8) C ? (G;V )§V Js - y °® Idy ; c°°(G;V )®V 

C'(G; V G ®V) Is ' Vo§{vls \ C°°(C7; V ®V) 
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commutes, where the vertical arrows are G-representation isomorphisms given by 

f®v^(g^ f(g)®g.v). 

The image of the bottom horizontal arrow of (8), to be viewed as a representation 
of G with the quotient topology, equals to the left-hand side of (7). The top horizontal 
arrow of (8) is the composition of the map 

(9) C'(G; V )®V -> (Indg V )®V 
and the map 

(10) (IndfVb)®^ -> C°°(G; V Q )®V. 

The continuous linear map (9) is surjective by [22, Proposition 43.9], and is then open 
by the open mapping theorem for Frechet spaces. The map (10) is injective since V is 
nuclear (cf. [22, Proposition 50.4]). Therefore, the image of the top horizontal arrow 
of (8) equals to the right-hand side of (7). This proves the lemma. □ 

3.3. Casselman— Wallach representations. Let G be an almost linear Nash group 
as before. Denote by D ? (G) the space of (complex valued) Schwartz densities on G. 
Recall that D ? (G) = C ? (Gr) dg, where dg is a left invariant Haar measure on G. It is 
an associative algebra under convolutions. 

Let V be a smooth Frechet representation of G of moderate growth. Then V is a 
D f (G)-module: 

(f(g)dg).v := / f(g)g.vdg, f G V(G), v G V. 

We say that V is a Casselman-Wallach representation of G if 

• every D ? (G)-submodule of V is closed in V, and 

• V is of finite length as an abstract D <? (G)-module. 

It is clear that a Casselman-Wallach representation of G is irreducible if and only if it is 
irreducible as an abstract D ? (Cr)-module. Furthermore, by the open mapping theorem 
for Frechet spaces, it is easy to see that if 

> V x > V 2 V 3 

is a topologically exact sequence of smooth Frechet representation of G of moderate 
growth, then V-i is a Casselman-Wallach representation if and only if both V\ and V3 
are so. 

When G is reductive, du Cloux proves that V is a Casselman-Wallach representation 
if and only if its underlying Harish-Chandra module is of finite length (cf. [5, Section 
3]), and Casselman and Wallach prove that every finite length Harish-Chandra module 
has a unique Casselman-Wallach representation as its globalization (cf. [3] and [24, 
Chapter 11]). 

We return to the setup in Section 2. Put 

k := {a G K | a' = a}, 
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which is either R or C. Let E be a skew- Hermit ian K-module as before. Put := E, 
to be viewed as a symplectic space over Ik under the form 

(u,v) E (v,u) e 
(u,v) Ek := — — ■ 

The associated Heisenberg group is defined to be 

H(E) := k x E, 

with group multiplication 

(a, v) ■ (a, v) := (a + a' + (?/, u) Ek , i; + i/) . 

The group U(E) acts (from left) on H(E) as group automorphisms through its nat- 
ural action on E. This defines a semidirect product (the Jacobi group) 

3(E) := E(E) x XJ(E), 

and its covering 

/(£) := B(E) x U'(£). 

They are almost linear Nash groups, both having H(E) as their unipotent radicals. 
Recall that k is the center of H(E). 

Let p be an irreducible Casselman-Wallach representation of I'(E). By a version 
of Schur Lemma ([5, Proposition 5.1.4]), k acts through a character ip p in p. We say 
that p is nondegenerate if ip p is nontrivial. Note that the moderate growth condition 
implies that ip p is unitary. Since all the Jacobi groups which occur in Theorem A and 
Theorem B are isomorphic (as Nash groups) to some f(E), we also get the notion of 
"nondegenerate irreducible Casselman-Wallach representations" for these groups. 

Fix a nontrivial unitary character tp-^ on k. Let u be a smooth oscillator represen- 
tation of I'(E) associated to it, namely, it is a Casselman-Wallach representation of 
I'(E), and when viewed as a representation of H(E'), it is irreducible with central char- 
acter tpfc. Smooth oscillator representations of f(E) exist by the well known result of 
splitting metaplectic covers ([14], see also [13, Proposition 4.1]). If 7r is a Casselman- 
Wallach representations of XJ'(E), to be viewed as a representation of S'(E) via inflation, 
then cu®7r is a Casselman-Wallach representations of f(E) so that k C f(E) acts by 
the character tp-^. Conversely, all such representations are of the form u^ttq for some 
ttq. Furthermore, u§nro is irreducible if and only if 7Tq is. See [20] for details. 



4. Reduction to the basic case 

We continue with the notation of Section 2. We reformulate Theorem A more pre- 
cisely as follows: 

Theorem 4.1. For every irreducible Casselman-Wallach representation 7r ofXJ'(E), 
every nondegenerate irreducible Casselman-Wallach representation o of J' X ,(E') , and 
every nondegenerate unitary character ip on N?(X), one has that 

dim Horn j' ( E ) (ir0a } iff) < 1. 
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In this section, we explain the strategy of the proof of Theorem 4.1. 

Fix two totally isotropic free submodules Y D Y r -\ of E such that the parings 

(,) E :IxF4l and ( , ) E : X r _i x Y r -i K 

are nondegenerate. Fix x r G X and y r G K so that 

(x r , Y t _i)e = 0, (X r _i, y r ) B = 0, and (x r , y r ) E = 1. 

Identify £" := X^JX^ with (X r _i © Y^i^, and £ := with (X © Y) 1 . 

Then we get decompositions 

E = X r _x © £" © F r _i = X © £ © Y and = Kx r © E © Ky r . 

Identify X' := X/X r _i C £" with Kx r , and J X i(E') with J(-E'o) via the isomorphism 

(11) g >-> ({gyr,y r )Ei, [gy r - y r ]E , [#]e )- 

Here for every v G Kx r © £?o, denote by [v]e G £?o the image of u under the projection 
Kx r © E'o — > E , and for every g G Jx'(E'), denote by \q\eq the element of \J(E ) so 
that the diagram 

g\Kx r (S)E () 

Kx r © £ ^ Kx r © £ 

[g] B Q 

-C'O ^0 

commutes. 

Fix the unique identification J' X ,(E') = f(E ) so that the squares in 

i — ^ K — - y x ,(E r ) — - j xi (e>) — - 1 



- J'(E ) - J(E ) - 1 

are commutative. 

Denote by P' x the parabolic subgroup of \J'(E) stabilizing X, and by M' x the Levi 
subgroup of U'{E) stabilizing both X and Y. Then we have a Levi decomposition 

P x = N x x M' x , 

where N x is the unipotent radical of P^. Moreover, we have 

, _ U'(E ) x GL'(X) 

X A/i K 

where GL'(X) is the subgroup of M' x fixing E pointwise, and A/xk is the group /xk 
diagonally embedded in U'(E ) x GL'(X). 

Put H(X J -) := Ik x X- 1 , which is a subgroup of H(E). Define a homomorphism 

p x : H(X-L) xP x H(£ ) *M X , 

(t,v + v ;uh) \-> (t,v ;h), 

where t G k, u G X, w G £ , it G N x and ft, G M x . The kernel of p x is X x N*. We 
always view H(i£o) * M x as a quotient of H(X ± ) xi P' x via the map px- 
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Since the covering map GL'(X) — > GL(X) uniquely splits over Ny(X), we also view 
Ngr(X) as a subgroup of GL'(X). 

The following lemma is routine to check and is the key to the proof of Theorem 4.1. 



Lemma 4.2. The following diagram 



(13) J'j{E) H(X ± ) x 



PA" 



f x ,(E') x N y (X) H(E ) x M' x 

commutes, where the bottom horizontal arrow is the map 

J X ,(E') x N?(X) c y x ,(E') x GL'(X) 

J' X ,{E') x GL'(X) f(E ) x GL'(X) 



H(£ ) x (U'(E ) x GL'(X)) 



E(E ) x . 



A/i K 

For the top horizontal arrow of (13), note that J' 9 (E) C P' x C H(X X ) x c 1(E), 
and y r <E E C E(E) C J'(£). 

As in Theorem 4.1, let a be a nondegenerate irreducible Casselman-Wallach rep- 
resentation I'(E Q ) = J' X ,(E'). Fix a generic irreducible Casselman-Wallach represen- 
tation r of GL'(X) so that Xr = X<x> where Xa is the character of /^k through which 
/iK C J'(-E'o) ac ts i n an d likewise for Xr- Then cr£g>r descends to a representation of 

J'(E ) x GL'(X) , 
= H(£ ) x M x . 

We recall some notations in [11]. Put 

1 if IK is a field; 



and 



2 otherwise, 



K* := (R*) dh (R* is the multiplicative group of positive real numbers). 



For all a£l" and s G C dK , put 

a s := a^ 1 a^ 2 G C x , if dx = 2, a = (ai, a 2 ), s = (si, s 2 ), 

if dg = 1, a s G C x retains the usual meaning. 

For every s G C dK , denote by r s the representation of GL'(X) which has the same 
underlying space as that of r, and has the action 

r s (g) = \g\ s r(g), g G GL'(X), 

where \g\ s is the image of g under the composition map 

GL'(X) -> GL(X) dctcrminant > K x 4l^C x , 
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and 



is the map of taking componentwise absolute values. Then o~®t s is an irreducible 
Casselman-Wallach representation of H(Eq) x M^. By inflation through px, we view 
it as an irreducible Casselman-Wallach representation of H(X ± ) x P^. For simplicity 
in notation, put 



I, 



T A$'( E ) 

U H(Vl),p/« 



)T,. 



Proposition 4.3. Except for a measure zero set of s G C dK , the unnormalized Schwartz 
induction l s is a nondegenerate irreducible Casselman-Wallach representation off(E). 

Proof. Assume that Ik C 3'(Eq) acts through the nontrivial unitary character ip^ in 
cr. Then Ik C f(E) also acts through ip^ in l s . As in Section 3.3, let w be a smooth 
oscillator representation of f(E) associated to 

Denote by ux the topological X-coinvariant space of u, namely, it is the maximal 
Hausdorff quotient of u on which X C H(£J) acts trivially. This is a representation of 
H(X ± ) x P' x . By using the mixed Schrodinger model (cf. [9], [12, Section 5]), we know 
that X xi Nx acts trivially on ux, and it descends to a smooth oscillator representation 
of B.(E ) x M^. By Frobenius reciprocity and using Schrodinger models, we know that 
the quotient map u — > Ux induces an isomorphism of H(i?) x P^-representations: 

(14) wlwrR^P'.. = Ind v 

Let seC dK . Put 



0J\ 



H(S)xP^ 



H(X^)><iP' 



Horn 



B(E ) 



equipped with the compact open topology, 
representation of M' x , and we have (cf. [20]) 

(15) a®T s = U X ®Qs 

as representations of H(E ) xi M^. 
Then as /(^-representation, 



It is an irreducible Casselman-Wallach 



L 



Ind 



'« ,(Ind»«-/ p ,,Sr, 



Ind H(i?)>,P^( Ind H(X^)^^®^) 

Ind H(i?)xP^(( Ind H(x^)>,P^^)®^) 



Ind 



f{E) / | - 
H^hP^ lH(£)xP^ 



w®Ind' 



f{E) 
H(E)*P^ 

w<g>lnd p / ' q s . 



Qs 



by (6) 
by (15) 

by Lemma 3.2 
by (14) 

by Lemma 3.2 



By using Langlands classification and the result of Speh-Vogan [18, Theorem 1.1], 
we know that except for a measure zero set of s e C dK , Ind"/ ; g s is an irreducible 
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Casselman-Wallach representation of XJ'(E). This finishes the proof by the argument 
in the last paragraph of Section 3.3. □ 

Fix a nonzero element A of the one-dimensional space Hom^ym (t, It induces 

a continuous linear map 

A: <7®t — > cr, u ® v i— )■ \(v) u. 

Let tt be an irreducible Casselman-Wallach representation of U'(E) as in Theorem 4.1, 
and let 

(, ) M : tt x a C 

be a continuous bilinear map which represents an element fi £ Homy ( E ) (7rcg)cr, iff). As 
before, let s £ C dK . For every / £ I s and u £ 7r, consider the following function on 
U'(£): 

(16) ^^(Ao/)^)),. 

Here, both g £ U'(i£) and y r £ E <Z H(E) are viewed as elements in J'(-E) = H(i?) x 
U'(-E), and / is viewed as a a<S>r- valued function (r s = r as vector spaces). It follows 
from Lemma 4.2 that the function (16) is left Jy(.E)-invariant. 
Put 

Z M (./» := / (9-U,(Ao ^(y^g^^dg, f £ I s , W £ 7T, 

^4(E)\U'(E) 

where dg is a fixed right U'(.E') -invariant positive Borel measure on J^(£')\ XJ'(E). 
We postpone the proof of the following result to Section 5. 

Proposition 4.4. Assume that /i ^ 0. Then for every s £ C dK , there are elements 
f £ Lj anc? m £ 7r stzc/i t/ie integral Z At (/, tt) zs absolutely convergent and nonzero. 

For every s £ C dK , denote by Res £ M dK its componentwise real part. We write 
Res > c for a real number c if every component of Res is > c. In Section 6, we prove 
the following 

Proposition 4.5. There is a real constant c M , depending on tt, a, r and fi, such that 
for every s £ C dK with Res > c^, the integral Z M (/, u) is absolutely convergent for 
every f £ l s and u £ tt, and defines a U'(E) -invariant continuous linear functional on 

I s <g>7T. 

Now we are ready to prove Theorem 4.1, as in the discussion of [11, Section 3.4]. 
Let F be a finite dimensional subspace of Homj' (#) (tt®o~, if). By Proposition 4.5, we 
have a linear map 

F ->■ Homu'(B) (l s ®7r, C) , i y Z M 

for Res > cf, where cf is a real constant depending on tt, a, r and F. Moreover, 
by Proposition 4.4, the above map is an injection. In view of Proposition 4.3, choose 
s such that Res > cf and l s is irreducible. Then dime Homy' (#) (l s <8W, C) < 1 by 
Theorem B. Therefore dime F < 1 and Theorem 4.1 is proved. 
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5. Proof of Proposition 4.4 

We continue with the notation of the last section. Denote by P' Y the parabolic 
subgroup of XJ'(E) stabilizing Y. It has a Levi decomposition 

P' Y = GL'(Y) x Ny, 

where Ny is the unipotent radical, and GL'(F) = GL'(X) is the subgroup of U'(-E') 
stabilizing X and Y and fixing E pointwise. Denote by P' (Y) the subgroup of GL'(Y) 
fixing y r , and by P' Yr _JX) ^ ne subgroup of GL'(F) fixing y r _i pointwise. Then the 
multiplication map 

is an open embedding, and its image has full measure in GL'(F). It is also routine to 
check that the map 

(rt (H(X^)^P^)x(P yri (y)KNy) 

(17) r -± >J(E), (g,h)^gy r 1 h 

is an open embedding. 

Take two vectors u a G a and u s G r s . Take a compactly supported smooth function 
<p on P yr i (r) x Ny so that 

(j){zh) = X r{z)(f>{h), ZEfl K ,hE P'y^Y) K Ny . 

Recall that cr®r s is a representation of H(£ , ) ^ M^-, and is viewed as a representation 
of H(X ± ) x P' x by inflation. Put 

0'(<7, h) := (f>(h) (g.{u a ® «,)) , <? G H(X ± ) >iP^G Py^OO x Ny . 

Extension by zero of <fi' through (17) yields a a®T s - valued smooth function / on J'(E). 
By Lemma 3.1, / G L. 

It is elementary to see that there is a positive smooth function r y r on (Nj(X) x 
^)\P' yr (Y) so that 

/ (p(g)dg= / ^ r (h)(p{hk) dhdk, 

Jr T {E)\\J'(E) J ((N y (X)x /lK )\ Pj, r (Y)) x (P' yr _ t (F)kNv ) 

for all nonnegative continuous functions y> on J^(£')\ U'(-E), where dh is a right P' (Y)- 
invariant positive Borel measure on (Nj(X) x ^)\P' yr (Y), and dk is a right invariant 
Haar measure on P' Y AY) x Ny. 



UNIQUENESS OF FOURIER-JACOBI MODELS: THE ARCHIMEDEAN CASE 13 
For every u G 7r, we have that 

(g-u, (A o f^iy^g))^ dg 

^(E)\V'(E) 

lr (h){(hk).u,(Ao f)( y ; l hk))^dhdk 

((Nr(X)x m )\P; r (Y))x(P^_ i (y)xN y ) 

^((h^.u^Aih.ifiy^km^dhdk 

((N 3 -(x)x m )\p; r (y))x(p^_ i (y)xN y ) 

7 r (/i) 4>{k)({hk).u, A(u a <g> Ts^fyus))^ dh dk 

((N 3 -(x)x m )\p; r (y))x(p^ r _ i (y)KN y ) 

\(r s (h)u s )&(h, k) dh dk, 

((N 3 .(x)x m )\p; r (y))x(p^_ i (y)xN y ) 
where 

$(M) := ~f r (h)<j>(k)((hk).u, u^, h G P^Y), fc G P^Y) x N y . 
Choose 0, m and m ct appropriately so that the function 

:= f $(h,k)dk 

does not vanish at 1. 

Note that the smooth function ^ on P' yr (Y) satisfies 

*(bzh) = il>(b)xr(zMh), b g N?(X), zefi K ,he P' Vr (Y). 

Recall from [10, Section 3] that for every smooth function W on P' yr (Y) such that 

Wibzh) = ^{b)- l XT {z)W{h), b g N?(X), zefi K ,he P' yr (Y), 

if W has compact support modulo N^X) x /j K , then there is a vector w' s G r s such 
that 

Therefore we may choose u s appropriately so that the function h h- > \(r s (h)u s ) on 
P' yr {Y) has compact support modulo Nj-(X) x /xk, and that 

A(r»(/i)« a )*(/i) d/i ^ 0. 

(N^(x)x m )\p; r (y) 

Note that the integral Z /1 (/, u) equals to the left-hand side of (18), and its integrant is 
smooth and compactly supported. This finishes the proof of Proposition 4.4. 
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6. Proof of Proposition 4.5 

Extend x r G X to a K-basis {xi, X2, • • • , x r } of X so that xi G Xj for i = 1, . . . , r. 
Under this basis, (K+) r embeds in GL(X): 



(K*) r c (K x ) r = JjGL(Kxi) c — >■ GL(X). 



i=l 



Since the covering GL'(X) — > GL(X) uniquely splits over (K^.) r , it also embeds in 
GL'(X). For every t = (t 1 ,t 2 , . . . , t r ) G (K*) r , denote by a t the corresponding element 
in GL'(X), and put 

r > — 1 . 

||t|| := WwiU + tr 1 ) and f(t) := JJtJ7(l ' ' 



i=i i=i 



where for every t G K + , 

w{t) :-- 



t if d K = 1; 

ft" if d K = 2 and t = (t',t"). 



Fix a maximal compact subgroup X of U'(E). It is elementary to see that there is 
a positive character 5 r on (K+) r such that 



(19) / y(g)dg = / 5 r (t) cp(a t k) d x t dk 

for all nonnegative continuous functions on Jj(i?)\ U'(-E'), where dk is the normalized 
Haar measure on X, and d x t is an appropriate Haar measure on (K+) r . Pick a positive 
constant c so that 

(20) S r {t) < \\t\\ C0 , tG(K x ) r . 
Recall that 

(, Tr x a -> C 

is a continuous bilinear map which represents an element fi G Homj^ E ) (tt<S)<j, tp). Pick 
a continuous seminorm | ■ on 7r and a continuous seminorm | • \ a on u so that 

(21) KWj^/xl < |w|tt,1 kU, U G 7T, f G C. 

By the moderate growth condition on 7r, there is a constant Ci > and a continuous 
seminorm | • \ n 2 on tt such that 

(22) |(atfc).u|^i < ||t|| Cl \u\ n>2 , t G (K x ) r , fc G X, u G tt. 
Recall that A G Hom N3 .(x)(r,^ _1 ) induces a continuous linear map 

A: o®t — > cr, M®«4 A(f ) It. 
Still denote by r the following continuous linear action of GL'(X) on a®r: 

r(h)(u <g> v ) '.= u ® (r(h)v), u G a, t> G r. 
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The moderate growth condition on r implies that there is a constant c<i > and a 
continuous seminorm | • | (7 g T on o~®t such that 

\A(r(a t )w)\ a < ||t|| C2 \w\ a§)T , t G (K x ) r , w G agr. 



Lemma 6.1. For every positive integer N, there is a continuous seminorm 
on a®r such that 

(23) \A(r(a t )w)\ a < £(t)~ N 1 1*1 \° 2 Hogr,*: * e (K*) r ', «; G agr. 

Proof. This is similar to the proof of [11, Lemma 6.2]. We omit the details. □ 

Put c M := c + Ci + c 2 . Recall that s 6 C 4 . 
Lemma 6.2. //Res > c M; i/ien t/iere zs a positive integer N such that 

(24) / ||t|| c -n(t) Rcs ^(t)- Af u7(l + t r )- 7V d x t < oo, 
where 

r 

(25) t = (tx,t 2 , ...,t r ) and n(t) := JJt 4 G K*. 

i=l 

Proof. We assume that dx = 1. The other case obviously follows from this one. Write 

ti+1 

Then 

' |tii<n[=i(^+«r 1 )' ) 

n(t) = n!=i4 

^(tMi+t r ) = ni =1 (i+« i ). 

Therefore the left-hand side of (24) is at most 

where d x «j is an appropriate Haar measure on M*, z = 1, 2, . . . , r. It is elementary to 
see that if N > r(c M + Res), then 

at + qT 1 )^ a* Res (1 + at)-* d x a; < oo, i = 1, 2, . . . , r. 

This finishes the proof. □ 

Now assume that Res > c M . Let / G L, to be viewed as a o~cg)T-valued function 
on J'(-E'), and let u G 7r. We want to show that the integral Z^(f,u) is absolutely 
convergent and defines a U'(-E)-invariant continuous linear functional of L (§>tt. The 
U'(i?)-invariance is obvious as soon as the absolutely convergence is proved. 
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We have 
\MM\ 

< / \(g-u, (Ao/Jrt^ 

J^(E)\V'(E) 



= [ 5 r (t)\((a t k).u, (Aof)( y ; 1 a t k)) ll \d x tdk by (19) 

J(K*yxK 

< [ \\t\\ C0 Ka^.ul^KAo f^a^dHdk by (20) and (21) 

J(M>*) r xK 

< [ ||tp+ c >M(Ao/)(^V£:)UdHdfc by (22). 



Let N be a positive integer as in Lemma 6.2, and let | • \ a §, T N be a continuous 
seminorm on cr£g>r as in Lemma 6.1. Then for every t = t 2 , ■ ■ - ,t r ) G (K+) r and 
k G K, we have 

l(Ao/)(y~ 1 o t A;)| ff 

= |n(t) s | |A(r(a t )/(a t - 1 2 / r : 1 atA;))| (T n(t) is as in (25) 

< n(t) Re ^(t)-^||t||^ \f(a^y; l a t k)U T>N by (23) 

= m Rcs m- N \\t\\ C2 \f((-try r )k)u T , N 

<n(t)* es t(t)- N \\t\\ c >w(i + t r )- N \f\ Is;N . 

Here 

\f\ Is>N := sup {w(l + tf \f((-ty r )k)U T>N \teKl,kEK}. 

It is easy to see that | • |i s ,jv is a continuous seminorm on I s . 
Therefore 



|z M (./»| < \f\i s , N \u\„ >2 [ \\t\\ c >n(t)^'z(t)- N w(i + t r )- N d*t, 

J(K*y 

and Proposition 4.5 follows by (24). 
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